Analyzing Variation of Adaptive Game-Based Training with Event Sequence Alignment and Clustering

Adaptive Game-Based Training (AGBT) systems plan the content and ordering of learning opportunities to customize game-play, thereby addressing individual players’ learning needs. Determining the best combination of settings, rules, and algorithms to perform this intelligent tutoring is both complex and expensive, as it requires iteration over multiple experimental cycles so as to measure learning performance and compare it with pre/post testing. In this paper, we propose analyzing event sequence variations produced by the intelligent tutor as an indicator of the effect of changes on adaptive game-play experiences. We describe Event Sequence Alignment and Clustering (ESAC), an analytic method that characterizes variations in the selection and ordering of learning opportunities directly from play-test game logs (without pre/post testing). We present results of a post-hoc analysis of variation, over three experimental cycles, on a large-scale AGBT development effort. We conclude with a discussion of limitations, applications, and future work.